Kidney Stone CNN — Clinical Evaluation Report

Model: EfficientNet-B4 with custom classification head
Date: February 21, 2026
Dataset: 12,446 images (CT + Ultrasound) | Test set: 1,904 images
Status: ✅ All clinical KPI targets exceeded

Key Performance Indicators

100.0%

Sensitivity
(Target ≥ 92%)

99.2%

Specificity
(Target ≥ 88%)

1.0000

AUC-ROC
(Target ≥ 0.95)

False Negatives
(Missed Stones)

Confusion Matrix & ROC Curve

The ROC curve shows perfect discrimination (AUC = 1.0). The confusion matrix confirms zero missed stone cases across all 224 stone images in the test set.

Grad-CAM Visual Explanations

Grad-CAM++ heatmaps show which image regions drove each prediction. Red/yellow areas = high model attention. These should correspond to kidney and urinary tract anatomy for valid predictions.

False Positive Analysis (14 cases)

All 14 false positives are shown below with Grad-CAM overlays. These are no_stone images incorrectly predicted as stone. Common causes: cysts mimicking stones, vascular calcifications, image compression artifacts.

Clinical impact: These 14 cases would trigger unnecessary follow-up imaging but would not cause harm. Zero false negatives means zero missed stones — which is the primary safety requirement.

Threshold Calibration

Decision threshold was optimised using F2-score on the validation set. F2 weights recall (sensitivity) twice as heavily as precision, appropriate for a screening tool where missing a stone is more harmful than a false alarm.

Detailed Metrics Table

Metric	Value	Target	Status
Sensitivity (Recall)	1.0000	≥ 0.92	✅ PASSED
Specificity	0.9917	≥ 0.88	✅ PASSED
AUC-ROC	1.0000	≥ 0.95	✅ PASSED
Precision	0.9412	≥ 0.85	✅ PASSED
F2-Score	0.9877	≥ 0.90	✅ PASSED
False Negatives	0	Minimise	✅ ZERO
False Positives	14	< 5% of negatives	✅ 0.83%

Known Limitations

Dataset is Kaggle CT + Ultrasound only — no TCIA DICOM data. Performance on different scanner manufacturers is unknown.
Patient-level split was not possible due to missing patient IDs in source dataset. Sequential CT slices may appear in both train and test.
Only 952 stone training images — model may underperform on rare stone variants (<3mm, faint calcifications).
AUC = 1.0 on the test set may indicate some data leakage via sequential CT slices. Results should be validated on an independent external dataset before clinical deployment.

Next Steps — Phase 4

Model is ready for API development and deployment. Phase 4 will wrap this model in a FastAPI REST endpoint, containerise with Docker, and serve predictions with Grad-CAM heatmaps via HTTP.

Generated automatically by scripts/generate_report.py · 2026-02-21 21:19 · Kidney Stone CNN v1.0

🫘 Kidney Stone Detection CNN — Clinical Evaluation Report